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Abstract 

The rapid growth of online learning has led to the development of faculty inservice evaluation 
models focused on quality improvement of degree programs. Based on current 'best practices' of 
student online assessment, the Online Faculty Development and Assessment System (OFDAS), 
created at the Canary Islands, was designed to serve the dual purpose of faculty development and 
classroom learning environment assessment. Results, as illustrated in this paper, show that the 
OFDAS encouraged faculty to reflect on the professionalism of their teaching skills. Implications 
arc discussed in terms of emphasizing the process of online teaching, knowledge acquisition, and 
incorporating varying perspectives, all which yielded a comprehensive view of faculty teaching 
attitudes and their relationship to student's perceptions of their classroom environment. 

Keywords: Faculty development; teaching professional skills; online student learning; 
environment assessment; online professional development 

Introduction 

Online Faculty Development and Evaluation System 

The Online Faculty Development and Assessment System (OFDAS) is a voluntary professional 
skill program. The course encompassed three components: 1) online faculty professional skill 
learning experiences; 2) faculty professional skills learning assessments; and 3) a student online 
learning environment assessment. 

The critical issues in designing OFDAS included, among others, planning, organizing, 
structuring, tracking, reporting, and communicating assessments - efforts which took time and 
required orderliness on the part of the online program advisers. Based on their experiences, we 
found the following five-stage design process to be a highly effective strategy: 

1 . Provision of a face-to-to face workshops (four hours long) 
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2. The design of sequenced, structured, and comprehensive lessons. The learning activities 
of these lessons sought to engage and direct the OFDAS faculty participants in the 
'process' of professional skills acquisition 

3. An online support system was established to help faculty scaffold their teaching and 
learning, provide a mechanism for targeted feedback from mentors, and provide a safe 
forum for sharing ideas and problems with other faculty participants 

4. Provision and coordination of learning resources to help faculty successfully complete 
their learning activities 

5. Provision of a detailed inventory of students online classroom assessments, which 
enabled faculty to reflect upon and access feedback on their teaching performance 

The OFDAS model recognizes faculty personal and professional objectives at various stages of 
their academic career. As such, it was necessary to create diverse training strategies. Accordingly, 
prior to the implementation of the OFDAS, the amount of time needed to complete the 
professional development program was negotiated with higher education institutions. A formal 
face-to-face workshop was delivered prior to starting the online course as well. Finally, in an 
attempt to ensure intrinsic motivation and to encourage participation among faculty, an official 
learning certificate was developed. 


Faculty Continuing Learning Opportunities: Skills development 

The authors developed a framework for training professional skills reflective of student-centred 
education (Villar, 2004); that is, a framework that focused student learning experiences and 
processes within the university social context (Badley, 2000). Prior to engaging in the OFDAS, it 
was expected that faculty have a deep understanding of their scientific field, as well as requisite 
pedagogical and didactic skills specific to their discipline. Accordingly, the concept of 
'professional skill’ was defined by the authors as: "an integrated set of knowledge, beliefs, 
abilities and attitudes that were basic for good performance in various university teaching 
settings. " Common elements in the OFDAS program were to develop faculty competence in the 
design of curriculum and course material, and help faculty acquire didactic and guidance skills 
(Tigelaar, Dolmans, Wolfhagen, & Van Der Vleuten, 2004). Three basic principles predominated 
in OFDAS: helping faculty understand that, 1) academics and students are different, thus 
curriculum and implementation of classroom methods must be designed to respect student 
diversity and identity; 2) professoriate are dependent on one another in collegial and classroom 
interactions; and 3) online faculty development courses increase one’s own decision-making 
processes and learning by online student assessment. Consequently, ten professional skills were 
proposed (see Figure 1). 

Online Assessment and Feedback 

Online assessment of classroom teaching and learning processes has been the focus of numerous 
studies that examine the degree this method of assessment influences and facilitates changes 
within learning organizations. Online assessment questionnaires should contain student 
demographic information such as age and gender, as well as other items seeking students’ 
opinions about the quality of the course (Lounsbury, Saudargas, Gibson, & Leong, 2005). 
Recommendations for the online classroom climate assessment has components similar to 
conventional assessments. For example, some scholars focused their attention on 'grading 
procedures' and were very explicit when such tasks, projects, and tests should be assigned 
(Summers, Waigandt, & Whittaker, 2005). 
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From an administrative point of view, research has been conducted examining 'why' online course 
evaluation should be used. According to this body of research, online course evaluations lower 
costs, allow more time for teaching, lessens administrative burden, and so on (Ballantyne, 2003). 
The question remaining for us, therefore, was whether online student feedback gathered from 
questionnaires (see Appendices A and B) actually does lead to improvements in teaching quality. 
Until recently, these types of feedback tools (i.e., questionnaires) have formed the basis upon 
which to compare teachers, departments, and even universities. However, although correlations 
have been found to exist between teaching quality and online student feedback, it does not 
necessarily imply causality, as other factors could also affect the quality of teaching over time 
(Kember, Leung, & Kwan, 2002). 

Literature on students’ assessment of classroom environments has been on the rise for over a 
decade (Aldridge & Fraser, 2000) and the field of learning environments has undergone 
remarkable "diversification and internationalisation" (Fraser, 1998, p. 7). Evidence (largely 
derived from on-demand university teaching quality assessments) has accrued regarding the 
potential of classroom learning environment assessments and their ability to improve university- 
level teaching and learning, as well as staff development (Dallimore, Hertenstein, & Platt, 2004). 
For example, classroom climate questionnaires administered in a field-specific class were found 
to result in reflective changes in learning and instruction (Wildman, Hable, Preston, & Magliaro, 
2000 ). 

Designing Powerful Interactive Learning Environments: The case of 
OFDAS 

Following are some key features of the online course used in this study: 

1. Faculty were given a professional skills handbook (Villar, 2004). This handbook (with 
specific focus paid to teaching innovation and student learning) reviewed several sources 
on college teaching, and identified critical professional skills related to class preparation, 
classroom structure, and organization. 

2. Materials were broken down into ten lessons and released weekly, with ongoing updates 
revolving around seven basic modules or dimensions, similar to the teaching 
competencies framework suggested by Tigelaar, Dolmans, Wolfhagen, and Van Der 
Vleuten (2004). The analytic structure of 'professional skill' comprised four phases: 1) 
purpose, 2) uses, 3) educational setting, and 4) case study. In total, 156 PDF and HTML 
documents, 114 websites, ten PowerPoint presentations, and over 500 glossary 
educational concepts and references, were published online and hyperlinked accordingly. 

3. Faculty discussed two topics in an online asynchronous forum: 'European Convergence 
issues,' and 'Students' efforts to cope with the new European credit system.' These topics 
were organized and released on a bi-weekly basis, and were accessible throughout the 
course. The final forum included postings on reflective questions (i.e., Socratic 
questions). Because we determined that 'participation' is crucial for learning, we built 
asynchronous online interaction into the design of the course. Indeed, when considering 
faculty postings to asynchronous discussions in online courses, Blignaut and Trollip 
(2003) remarked: "Determining the elements of faculty participation and involvement can 
lead to the development of improved skills, which in turn may lead to improved learner 
satisfaction, instructor satisfaction, and the lowering of attrition rates" (p. 153). 

4. Faculty accessed email from a central course website, which enabled one-on-one 
interactions with mentors and other participants. 
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5. Faculty had access to online curriculum materials hyperlinked to related articles and 
institutions, notes, and grades. 

6. Faculty could also download PowerPoint presentations, key concept maps, study guides, 
and other resources, etc. 

7. Faculty submitted their learning activity assignments online using the central course web- 
interface or via email. All course assignments, which presented faculty with complex 
teaching and learning tasks, were designed to be meaningful activities that had real-life 
relevance. 

8. Faculty engaged in activities that were designed to be 'realistic representations' of the 
tasks that the authors wanted to evaluate. As such, we allowed faculty substantial 
freedom in selecting activities, which according to Uhlenbeck, Verloop, and Beijaard 
(2002) are features of authentic assessment. 

9. Faculty completed ten online tests. Each 'professional skill' test was programmed 
(random selection) to be unique and to provide faculty instant feedback detailing their 
results on various tests. Faculty also assessed the quality of materials and the training 
process. In other words, an authentic assessment was woven into the course design, and 
provided faculty with a formative assessment of their understanding of the basic concepts 
covered in the course. This allowed faculty to make sense of their overall progress 
throughout the course. 

10. Faculty evaluated the OFDAS using the Attitude Towards Course Learning 
Questionnaire (ACLQ). 

11. Faculty assessed data on student classroom learning environments using the University 
Teaching Activities Questionnaire (AUTAQ). 

12. Similarly, faculty also received student feedback via the AUTAQ. 

(see http://gid.us.es:8083) : 


Research 


Research Questions 

Our broad research question was: "Plow did the online course elements and management 
activities affect professional skills learning and what were their impact on teaching attitudes and 
on the students’ classroom learning environment assessment?" This question was examined in 
light of three specific questions (see Figure 1). First, we asked was there a difference in faculty 
opinion about the quality of the online course? Second, did faculty learn professional skills? And 
third, after the course ended, was there a relationship between faculty teaching attitudes and 
students’ assessment of their learning environment? 


Villar & Alegre - Online Faculty Development and Assessment System (OFDAS) 


5 


Figure 1 . The OFDAS model and sample variables 
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Sample 

Twenty-four {n - 24) faculty members volunteered for this study and all met the following 
selection criteria: (a) employees of a university campus, (b) scientific field, and (c) professional 
merit. All respondents were full-time faculty employed by two public Canarian Universities: 1 1 
from La Laguna (45.8%) and 13 from Las Palmas de Gran Canaria (54.2%). Of the 24 faculty 
respondents, ten (41.7%) were male and 14 (58.3%) were female. Nineteen (n - 19; 79.2%) 
respondents were considered 'experts' (i.e., with more than five years of teaching experience). 
Most had a doctorate (n = 14; 58.3%). When disciplines were broken down into scientific fields, 
eight respondents (33%) reported that they taught in the social sciences; five (20.8%) in 
experimental sciences; four (16.7%) in healthcare sciences; three (12.5%) in humanities, and four 
(16.7%) in technical sciences. The OFDAS program took place during the spring quarter of the 
2006 academic year and spanned an 1 1 week period. 

The AUTAQ was administered to 102 students enrolled in courses the two universities taught by 
the faculty listed above. Four hundred eighty (n = 480) students were asked to provide feedback 
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on their classroom climate. The student sample was representative of gender, age, area of study, 
level of study, and other academic and social characteristics. 


Measures, Data Collection, and Analytical Methods 
Professional Skills Scale 

The Professional Skills Scale was used to measure respondents’ ability to understand the 
knowledge and skill acquisition levels, and the degree to which individuals or groups wished to 
use them (Cronbach's alpha = 0.944). Consisting of ten items used to measure 'expert skills,' 
'conditions,' 'technologies' and 'teaching practices' (i.e., "The competence was relevant for my 
teaching"), this scale was designed to measure faculty's knowledge, skills, and attitudes. For 
items 1 to 17, a five point Likert-like scale was used: 1 = strongly agree; 2 = agree; 3 = average; 4 
= disagree; 5 = strongly disagree. Items 8 to 10 had specific five point scales. All ten items 
measured various types of faculty 'opinions' over the duration of the course, such as professional 
skills relevance, usefulness, appropriateness, adaptation, tips, structure, pertinence, reading, 
impact, and time-consuming. This measure was developed for use in this study. 


Attitude Towards Course Learning Questionnaire (ACLQ) 


Faculty were asked to make 'attitude judgments’ on teaching practice (Cronbach’s alpha = 0.950). 
The measure consisted of 20 items (see Table 1, and Appendices A and B) scored on a 5-point 
Likert-type scale, ranging from 1 = ‘strongly agree,’ to 5 = ‘strongly disagree.’ The measure was 
developed for use in this study. Each scale consisted of four items. 


Table 1 . Description of Scales and a Sample Item for Each Scale of the ACLQ 


Scale 


Description 


Sample item 


Understanding 


Learning 


Extent to which faculty ate able to re- 
conceptualize, explain and use 
received information about teaching 

Extent to which faculty acquire 
knowledge, skills, attitudes, or 
values, through study, experience, or 
teaching, which lead to behavioral 
changes that are persistent, 
measurable, and specified 


I take time to understand 
the aspects of my teaching 
in which I am mistaken 

I discuss mistakes on 
authors’ articles and books 
that I read about teaching 


Discussion Extent to which faculty use a method 

of interaction and poation 
representational argument regarding 
Extent to which' faculty use a method 
of interaction and position 
representational argument regarding 
teaching 


Extent to which faculty agree on 
courses of action to tate in teaching 

Extent to which faculty determine the 


' educa&ohaf rational e* 


I point out my colleagues’ 
teaching weaknesses to 

help them clarify their 

. . . . fcs 

1 point out my colleagues 
teaching weaknesses to 
help them clarify 7 their 
educational rationale 


I share odd opinions about 
teaching with colleagues 


I re -- 


Negotiation 


N’ego tiatic 

Evaluatic 

teaching 

Extent to \ 
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Assessment of University Teaching Activities Questionnaire (AUTAQ) 


The AUTAQ was designed to appraise students’ perceptions of their classroom environment 
(Cronbach's alpha = 0.958). This questionnaire consisted of 22 items scored on a 5-point Likert- 
type scale, ranging from 1 ‘strongly agree’ to 5 ‘strongly disagree’ (see Table 2, and the 
Appendix). The design of the AUTAQ was guided by relationship, personal growth, and 
curriculum change dimensions for conceptualizing university quality assurance (Villar, 2001). 

Table 2. Description of Scales and a Sample Item for Each Scale of the AUTAQ 


Scale 

Motivation 

lamt 


Description 


Sample item 


>amp 

Extent to which university students I an 1 - fS ^ nei P 
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(4 items) 


Extent to which conjecture, 
questioning, and discussion in 
activities are fostered, and to which 
students socially interact with each 
other to give meanings to and reach 
agreements on teaching activities and 
viewpoints 


These activ 
university s 
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answers giv 
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(2 items) 


Extent to which university students 
are given explanations, examples and 
multiple forms of understanding a 
problem or difficult material 
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changed my views on the 
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These activ 
rtpvpl on oth 

(5 items) 


Involven 
(4 items) 


capacities in university 
students (e.g. handling of 
tools, document search, 
library' use) 



Data was collected online during and after the course. Faculty members explained to students the 
purpose of the AUTAQ and the research study, and assured them of full anonymity to encourage 
their participation. 
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Data analyses included descriptive statistical summaries, Alpha reliabilities of subscales of the 
two questionnaires, T-tests, analysis of variance (ANOVA), and intercorrelations among scales 
(Pearson product moment correlation coefficient). 

Results 


Research Question 1 

Research Question 1 asked whether faculty opinion towards the quality of OFDAS was positive. 
In terms of the professional skills quality scale items, item means ranged from a high of 3.08 
(Item 8, Reading: "I read websites and pdf documents which were linked to the professional 
skill") to 1.33 (Item 1, Relevance: "The competence was relevant for my teaching’). Standard 
deviations varied from 1.52 ( Item 8, Reading ) to .76 ( Item 1, Relevance ). All item mean scores 
exceeded the midpoint scale (3.00, normal), and Item 8, Reading, exceeded the midpoint scale 
(3.00, frequently). 

T tests revealed significant differences with regard to gender in five quality items (usefulness, 
adaptation, tips, structure, and pertinence). Females held better opinions than males with respects 
to the quality of the professinal skills. As to degrees, significant difference was found in eight 
quality items (relevance, usefulness, appropriateness, adaptation, tips, structure, pertinence, and 
time-consuming). PhD prepared instructors/ teachers held better opinions than Bachelor's 
prepared instructors with respect to the quality of professional skills. With regard to the degree of 
teaching expertise, new faculty (those with less than four years teaching experience) and expert 
faculty (those with five years or more teaching experience) had different opinions with respect to 
five quality items (usefulness, appropriateness, adaptation, tips, and structure) (See Table 3). 

Table 3. Significant t-Test Results for Demographic and Academic Factor Comparisons 
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Research Question 2 

Research Question 2 asked whether the online course stimulated faculty learning. This question 
was divided into two sub-categories: 1) facilitating learning activities, and 2) assessing the 
cognitive domain of professional skills learning. 

Facilitating Learning Activities: Oliver and Herrington (2003) assert: "Designing a learning 
environment by commencing with the design of learning activities creates a setting where the 
focus of the planning centres on formulating the forms of learning outcomes being sought, rather 
than considering what content will be covered" (p. 1 14). 

Learning activities were developed to reflect the manner in which curriculum and didactic 
knowledge will be used in real-life university environments. Descriptive summaries detailed the 
ten professional skills being assessed during the learning activities. Results show that faculty 
respondents completed 1,587 learning activities (see Figure 2). 

A principle of the learning process was peer assistance and peer review, which was provided via 
guidance and participant feedback. Online help was often needed. Thus, coaching and scaffolding 
of learning was provided by two OFDAS leaders assigned to diagnosed the strengths and 
weaknesses of each participant, and tailor any support needed. Figure 2 shows participant 
instructors’ changes in their interest in, and willingness to, respond to learning activities as the 
course progressed over the 1 1 week duration. Participation in the learning activities was found to 
be more intense in the earlier stages of the course, than during the final activities. Data collected 
shows that participants’ time commitment was not equally distributed. While participants engaged 
heavily in Professional Skill 2 (awareness of students’ diversity in all its forms), they engaged 
very little in Professional Skill 6 (knowledge of area being supervised - i.e., learning tasks, 
research, assessment, etc.). The last module on 'evaluation' saw low rates of participation 
(Professional Skill 9 and Professional Skill 10). In spite of the ebb and flow of participation, 
learning was fluent as faculty participants were made aware of new possibilities concerning their 
teaching. 

Figure 2. Participant Instructors’ Responses to Learning Activities 



Content analysis was chosen as a methodology for analysing the online faculty learning activities; 
this involved comparing and contrasting the activities using a 10 point scale to code and interpret 
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the results. Activity transcripts were scored by the researchers, and the scores were then actively 
discussed to arrive at a final version where the learning activities had been brought into 
alignment. Learning activities were scored as: “Maximum Distinction (9-10),” “Important for its 
Intensity (7-8),” “Suitable (5-6),” “Minimum Qualification (3-4),” and “Differed the Execution 
(0-2).” Highlights are provided in Figure 3, which demonstrates the ability of participant 
instructors to apply previously learnt solutions to learning activities. All 'professional skills’ were 
passed by participants, with the exception of the Evaluation Module (Professional Skill 9, 
knowledge of formative and summative evaluation, and Professional Skill 10, competence to 
conduct own self-assessment process). 

Figure 3. Learning Activity Qualifications 



Assessing the cognitive domain of professional skills learning: Ten tests (10 multiple choice 
items) measured participants’ knowledge and understanding of 'professional skills.' Test means 
varied from a high score of 7.4 (knowledge of student motivation and ability to promote students’ 
positive attitudes) to a very high score of 10 (teaching and didactic skills for large groups). All ten 
test means exceeded score 7 on the ten-point scale used. 
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Figure 4. Self-Assessment Test Scores 



Means and standard deviations on the ten self-assessment test scores are shown in Figure 4. It 
was found that faculty participants’ learning was effective. However, objective testing of 
professional skills showed that faculty participants’ performance was more effective in the 
'competence of teaching and didactic skills for large groups,' than in the 'competence of 
knowledge of area being supervised' (i.e., learning tasks, research, assessment, etc.). There were 
significant differences in the learning of Professional Skill 3 (competence to solve students’ 
problems) between participants in regards to gender (t (15) = 2.520, p = .018). Female instructors 
had more successful results than male instructors. Also, significant differences were found 
between instructors with and without previous educational knowledge in Professional Skill 1 
(knowledge of student motivation and ability to promote students’ positive attitudes) ( t (15) = - 
3.1 19, p = .008), Professional Skill 3 (competence to solve students’ problems) ( t (15) = -2.477, p 
=.027), Professional Skill 4 (competence to develop meta-cognitive skills in the trainee) ( t (15) = 
-2.385, p =.032), Professional Skill 7 (teaching and didactic skills for large groups) ( t (15) = - 
2.449, p =.028), and Professional Skill 8 (knowledge of questioning skills) (t (15) = -2.590, p 
=.022). All were in favor of the instructors with previous educational knowledge. Finally, in 
terms of measuring teaching experience, significant differences in learning Professional Skills 3 
(competence to solve students’ problems) were found between new and expert faculty 
participants (t (15) = 2.800, p =.015). 

Research Question 3 

Research Question 3 asked if was there a relationship between faculty members' teaching 
attitudes and students’ perceptions of their learning environment after completion of the OFDAS. 

Differences between students’ perceptions of actual and preferred environments: Table 4 
shows means, standard deviations, and a series of t tests used for comparison between the two 
AUTAQ actual and preferred forms. The findings revealed some clear patterns of differences in 
the suitability of the current classroom environments. The scale for classroom climate showed the 
highest means in both forms. 
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Table 4. Means, Standard Deviations, and ?- Values for Actual and Preferred forms of AUTAQ 


Scale 

Means 

Standard 

Dexiations 

t 

Motivation 

Actual = 4.53 

.637 

t(78) = 63.21,/x.OOl 


Pref. = 3.82 

.996 


Involvement 

Actual = 4.46 

.657 

t(78)= 60.448,/x.001 


Pref. = 3.83 

1.01 


Scaffolding 

Actual = 439 

.657 

t(78)=43.497^<.001 


Pref = 3.83 

1.01 


Climate 

Actual =4.74 

.609 

t(78)=69269^<.001 


Pref. = 4.74 

.518 


Clarification 

Actual = 4.40 

.689 

t(78)=56.SlS^<.001 


Pref = 353 

.991 



Use of Resources Actual =3.784 1.117 t(78)=30.1 13^.001 



Figure 5 shows the comparison between students’ actual and preferred forms. Students were 
found to be more satisfied with the actual classroom environment scales, than what they 
perceived from the other scales in the preferred classroom environment form. Moreover, climate 
scale means were the same in the actual and preferred forms. 

Figure 5. Significant Differences Between Student Actual and Student Preferred Perceptions of 
the AUTAQ 
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Table 5. Between Scale Correlations Calculated on Items Grouped into their Initial Tentative 
Scales 


Means. Standard Dev iations and rvalues for Actual and Preferred 
forms of AUTAQ 


Scale Means Standard Deviations t (78) 


Motivation Actual =4.53 .637 63.21./K.001 

Pie f. = 3.82 .996 

Involvement Actual = 4.46 .657 60.448, /xOOl 



Discussion 

This study was designed to assess faculty’s professional teaching skills, which are believed to be 
useful for gaining a better understanding of teaching practice. The aim was to develop and 
validate a framework of professional skills taught in an online program. The three research 
questions are discussed below. 


Opinion Towards the Quality of the OFDAS 

Faculty agreed that the professional skills taught in the OFDAS had a positive impact on their 
teaching skills. The leaders also had a positive impact on participants’ appreciation of the course 
and skills acquisition. This finding is supported by other researchers who have evaluated Web- 
based courses (Nijhuis & Collis, 2003). Based on gender, degree, and teaching experience, 
faculty gave different opinions on the usefulness of professional skills in terms of subject matter, 
resources, presentations, useful tips, learning goals for skills improvement, and the structure of 
these professional skills (purpose, uses, educational setting, and case study) for identifying, 
clarifying, and exploring educational situations. Thus, while the first research question was fully 
supported, it was found that participants’ opinions varied on the usage and ease of use of 
professional skills taught in the OFDAS. 


Learning Activities 

Faculty completed a total of 1,587 learning activities over the 11 week duration of the OFDAS. 
Faculty reported that the learning activities to be generally useful. This finding answers a 
question raised by Caffarella and Zinn (1999): "Do professional development activities assist in a 
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faculty member's professional success?" (p. 253). Based on this outcomes of the OFDAS, our 
answer to this question is a resounding "yes." Indeed, all but two professional skills were 
approved by faculty: the Evaluation Module (Professional Skill 9, knowledge of formative and 
summative evaluation, and Professional Skill 10, competence to conduct own self-assessment 
process). All ten Professional Skills test means exceeded 7 on the ten-point scale used. This 
finding supports our hypothesis that faculty can successfully acquire professional skills using an 
online program. Flowever, it should be reiterated that significant differences in 'learning' was 
found between participants in three nominal variables: gender, previous educational knowledge, 
and teaching experience. 

Relationship Between Faculty’s Teaching Attitudes and Students’ 
Perceptions of their Learning Environment 

The results stressed two somewhat different but conceptually related measures, which brought 
about new perspectives on assessing learning environments in higher education settings. The 
Climate scale in particular emphasized the importance of developing mature, interpersonal 
relationships, friendships, social bonds, and connections with other students, as a vector of 
behaviour of student development (Lounsbury, Saudargas, Gibson, & Leong, 2005). Results from 
the AUTAQ went to each faculty participant, just as Kember, Leung, and Kwan (2002) had done 
with the student feedback questionnaire used in their study. 

Overall, the findings of this study are encouraging. We found that all 'professional skills' taught 
were perceived by faculty to be useful and easy to implement, though at varying levels depending 
on the elements employed for the course (i.e., time available). As Fitzgibbon and Jones (2004) 
previously noted, the coordination of the online program is crucial to its success. Directed and 
purposeful course design efforts, coupled with activities best suited for various scientific fields, 
faculty are able to enhance their subject area teaching, with professional pedagogical skills that 
are easy to use and more importantly, used. 

The Attitude Towards Course Learning Questionnaire (ACLQ); the constructivist-based format of 
the Assessment of University Teaching Activities Questionnaire (AUTAQ); and the other faculty 
demographic and academic variables explored in this study, all addressed ongoing concerns about 
the need to improve online training in higher education, as well as emphasizing new ideas about 
important variables that might be measured as alternatives to the more traditional approaches in 
evaluation of faculty development (Ellett, Loup, Culross, McMullen, & Rugutt, 1997). The 
ACLQ and AUTAQ online systems facilitated timely data collection, feedback, and online 
assessment, a finding supported by previous research undertaken by Tucker, Jones, Straker, and 
Cole (2003). Finally, the AUTAQ consisted of two sections (although only Section II was used 
for this study). Section I collected student demographic, academic and social information, but was 
not used — as had occurred in the Barfield (2003) study. 

An issue arose from this study regarding student online assessment. Our response rates were low, 
a finding that is supported by previous research by Ballantyne (2003). Nonetheless, faculty 
reported that collecting feedback online with the AUTAQ system to be convenient, a finding that 
is supported by earlier research undertaken by Bullock (2003) who investigated a similar online 
feedback system. Based on this research, we opine that a good starting point for training to 
enhance faculty scholarship of teaching is built into the design of the OFDAS. In short, the 
OFDAS is a good starting point for this type of training, because it encourages faculty to become 
fully involved in the development of online faculty courses. It achieves this by inclusion of 
learning materials faculty themselves deem necessary to their learning within the scope of the 
face-to-face workshop, activities, quizzes, grades, and in the direction dialog takes within the 
OFDAS forum. The selection of 1 10 learning activities was the organising element of the user/ 


Villar & Alegre - Online Faculty Development and Assessment System (OFDAS) 


15 


faculty development-design process of the program. Our focus on learning tasks had also been 
already underlined by other researchers (Oliver & Herrington, 2003). 

One of the limitations found in this study was the fact that it examined solely one faculty online 
course at only two public urban universities in the Canary Islands. Because faculty volunteered 
their classes, our pool of respondents were not randomly selected. For these reasons, the study is 
not intended to be, nor should it be, generalized to other universities. 

Conclusion 

Based on our experiences at two Canarian universities, we opine that the OFDAS is an effective 
training model that can be used to improve reflective practice on professional skills. The OFDAS 
online system is a mechanism used to both enhance online faculty development program 
management and provide evidence of a quality-improvement process in such online development. 
Because professional skills framework had been field-tested in this study, faculty knew the 
impact of professional skills. Finally, no correlation was found between faculty’s teaching 
attitudes and students’ learning environment at the conclusion of the course. 
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Appendix 1 

Assessment of University Teaching Activities Questionnaire (AUTAQ) 

Instructions: This questionnaire is about your perception of the classroom learning environment. 
Your opinion is required for each question. For each sentence select the score that best suits your 
perception. Please answer by circling the number with 1 = ‘strongly disagree’, 2 = ‘disagree’, 3 = 
‘neutral’, 4 = ‘agree’ and 5 = ‘strongly agree’. 


Dimension A. MOTIVATION (extent to which University students are provided with 
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Appendix 2 (see next page ) 

Attitude Towards Course Learning Questionnaire (ACLQ) 

Instructions : This questionnaire is about your attitude towards the online learning course. Your 
opinion is required for each question. For each sentence select the score that best suits your 
attitude. Please answer by circling the number with 1 = ‘strongly disagree’, 2 = ‘disagree’, 3 = 
‘neutral’, 4 = ‘agree’ and 5 = ‘strongly agree’. 
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Dimension A. 
reconceptualise, 

UNDERSTANDING (extent to which faculty are able to 
explain and use received information on teaching). 

1. I invest time in understanding the aspects of my teaching in which I might be 
mistaken. 

Strongly 

Disagree 

Disagree 

Neutral Agree || Strongly Agree 


